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Abstract 

The spin-1 Heisenberg antiferromagnet on a triangular lattice with the nearest- and 
next-nearest-neighbor couplings, Ji = {1 — p)J and J2 = pJ, J > 0, is studied 
in the entire range of the parameter p. Mori's projection operator technique is 
used as a method which retains the rotation symmetry of spin components and 
does not anticipate any magnetic ordering. For zero temperature four second-order 
phase transitions are observed. At p ~ 0.038 the ground state is transformed from 
the long-range ordered 120° spin structure into a state with short-range ordering, 
which in its turn is changed to a long-range ordered state with the ordering vector 
Q' = ^0, at p 0.2. For p 0.5 a new transition to a state with a short- 

range order occurs. This state has a large correlation length which continuously 
grows with p until the establishment of a long-range order happens at p ~ 0.65. 
In the range 0.5 < p < 0.96, the ordering vector is incommensurate. With growing 

p it moves along the line Q' — to the point Qi = (O, — ) which is reached 



3^3; 

at p ~ 0.96. The obtained state with a long-range order can be conceived as three 
interpenetrating sublattices with the 120° spin structure on each of them. 
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The active interest in the two-dimensional Heisenberg model on a triangular lattice is caused 
by the possibility of an unconventional spin ordering in it. The calculations carried out for 
the classical spins and in the spin-wave approximation [T||2] demonstrated the rich phase 
diagram of the model when the nearest-neighbor coupling of spins is supplemented with 
more distant interactions. The interest in the model was revived by the recent synthesis 
of layered antiferromagnets NiGa2S4 [3] and AgNi02 In these quasi-two dimensional 
crystals, magnetic properties are mainly determined by Ni^+ ions with the spin S = 1. 
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These ions form a triangular lattice. NiGa2S4 demonstrates a spin disorder down to the 
temperature T ?a 0.35 K, incommensurate short-range spin correlations and a quadratic 
temperature dependence of the specific heat. 

In this article we consider the S = 1 Heisenberg model on a triangular lattice, taking into 
account the nearest- (NN, Ji > 0) and the next-nearest-neighbor (NNN, J2 > 0) couplings. 
We use Mori's projection operator technique [5] which retains the rotation symmetry of 
spin components and does not anticipate any magnetic ordering. In this approach, the spin 
Green's function is represented by a continued fraction. The elements of the fraction are 
calculated in the recursive procedure which is similar to Lanczos' orthogonalization [B]. We 
found that the 120° spin structure, two-sublattice and incommensurate phases with the long- 
range order (LRO), obtained for classical spins and in the spin- wave approximation [T|2] . 
are separated by phases with a short-range order (SRO). Thus, the phase diagram of the 
model appears to be even richer than that obtained before - it contains four second-order 
transitions. 



The Hamiltonian of the model reads 



where and are the components of the spin-1 operators Sn, n and m label sites of the 
triangular lattice, cr = ±1. The spin-1 operators can be written as = J2a=±i '^l^^; ^\ 
and = A/2(|n, 0)(n, — a| + |n, a){n, 0|), where |n, ±1) and |n, 0) are site states with different 
spin projections. As mentioned above, we take into account the NN and NNN interactions, 
-^nm = Ji J2a <^n,m+a + J2 J2d Sn,m+d with the vcctors a and d connecting the NN and NNN 
sites. By analogy with Ref. [7], where a similar model for 5 = | on a square lattice was 
considered, the frustration parameter p is introduced, Ji = (1 — p)J, J2 = pJ, J = Ji + J2- 
In the following we use J as the unit of energy and the lattice spacing as the unit of length. 

The retarded Green's function reads 

D{M) = -iemsm,s^_^]), (2) 



where sl^ = N Xln ^ ''^"s^, is the number of sites, s^(t) = e'-'^^s^e '^^^ and the angular 
brackets denote the statistical averaging. 

We exploit Mori's projection operator technique |5|6] for calculating the Fourier transform 
of Kubo's relaxation function, 
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The Fourier transform of Green's function ([2]) can be obtained from this relaxation function 
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using the relation 

Diku)=u{{si\s\%-{sls\), (3) 



where {A,B) = i dt{[A(t), B]). In this approach, is represented as the contin- 

ued fraction 

= [fkl^. , (4) 



u — El 



LO — hi 



where the elements En and Vn of the fraction are determined from the recursive procedure 

[An, H] = EnAn + An+l + K-l^n-l, K = (K, H], AI) 

(5) 

Vn-i = {An,Al){An-i,Ai_i)-\ V.i = 0, = 4, « = 0, 1, 2, . . . 

The operators A^ constructed in this procedure form an orthogonal set, {Ai,Aj) oc 6ij. 
Using procedure we get 



Eo = (tsl, s^-k)(4, slk)"' = ([4, s-k])(4, s-k)"' =0, A = ^sl 
Vo = 6J[(1 -p)Ci(7k - 1) +pC^d(7k - 1)] K,s\r\ 
El = i^Sl -ts\)itsl -ts^_^)-' = 0, 

where 7k = |cos(fc^.) + § cos(^) cos(^) and 7^, = ^cosiVSky) + | cos(^) cos(^) in 
the orthogonal system of coordinates, Ci = {s^^s^^^} and Cd = (Sn^s^^d) spin 
correlations on the NN and NNN sites, respectively. At this point we interrupt the continued 
fraction and calculate (s^, sij^.). In the accepted approximation Vi oc {A2, A^) = 0. From this 
equation we find 

([^'^k,-^^-k]) = 36J2[(1 -p)Ci(7k - l)+pCdi7L - l)]'(4,s-k)-'- (6) 



m 



The quantity z^s^ the left-hand side of this equation is a sum of terms of the type sf s^^s ^ 
Following Refs. p[9] . we use the decoupling 

4 



for the case 1 7^ m, n. Here a is the vertex correction. In contrast to the case S = ^ [10] , 
the terms with 1 = n or 1 = m do not cancel each other completely. For S = 1 the residual 
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terms read 



Pi = -= 5] 11 +1) (1, 0|s^ - |m, +1) (m, 0|sr - s+|m, 0) (m, +1| + s+ |1, 0) (1, +1 

We neglect these terms in following calculations taking into account that (Pi) = 0. 

Using this approximation for Ps^, from Eq. ([6]) we find (s|, sl^) and from Eqs. ([3]) and 

we get 



D(kuj) 



6J[(l-p)(7k-l)gi+p(7^,-l)Cj 
a;2 - ul 



(7) 



where 



u;^ = 36j2a<|(l-p)2(7k-l) 
'C, 



" _L r-' 2(1 - a) 



+ P(l - P) [(1 - 7k) {C" - liC) + (1 - 7L) (C^" - 7kC.)] 



C2 = l{l + l{st's-U^) + C, + Ci), C" = I ((4'^„Ud) + (4'^ni2a) + ^i) and = 
I (l ~'~ |('^n"'^'^n+2d) + C'd + (s^ ""^s^^ga)^ . As follows from Eq. ([7]), the quantity is the fre- 
quency of the spin excitations. From Eq. ([8]) we see that this frequency tends to zero when 
k ^ 0. 

To find the parameters a, Ci, C2, C'2, Cd and C" in Eqs. ([7]) and ([H]) we use the relations 
connecting the spin correlations with Green's function ([7]) 



01 ^ g^k(n-m) (l-p)(7k-l)gi+p(7(,-l)C, 



coth 



Wk 



Wk 

2T 



(9) 



Five equations for Ci, C2, C2, Cd, C" which are derived from Eq. ([9]), and the equation 
("^m '^m ) — 4/3 , which follows from the constraint (s^) = 2, form the closed set of equations 
for calculating these parameters. 



The calculations were carried out for the entire range of the frustration parameter < p < 1 
and for the temperatures T/J = and 0.2 in a 100 x 100 lattice with periodic boundary 
conditions (for the case J2 = lattices up to 1000 x 1000 sites were applied). For the 
solution of the mentioned set of six equations we used the Optimization toolbox of the 
Matlab package. 
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Let us first consider the case J2 = (j9 = 0). In this situation, the above equations do not 
contain Cg, Cd and C" . For T = the system has LRO, which manifests itself in vanishing 
frequencies of spin excitations at nonzero ordering vectors. As follows from Eq. ([H]), these 
ordering vectors are Q = (^^, 0^ and (^^, — in the Wigner-Seitz cell (one of these vectors 
is indicated in Fig. 1; other four vectors of the corners of the cell belong to neighboring cells). 
These vectors correspond to the mentioned 120° spin structure. In the case of the LRO the 
parameters a, Ci and C2 are supplemented with a new parameter - the condensation part C . 
The equation for its determination is the condition = 0. Solving this set of equations we 

found C ~ 0.15, which corresponds to the sublattice magnetization m = ^3C/2 ^ 0.47. For 
this case the dispersion of spin excitations is shown in Fig. 2 (a). Thus the ground state of 
the S = 1 model with NN interactions is characterized by the LRO. In accordance with the 
Mermin- Wagner theorem [11] for nonzero temperatures the LRO changes into SRO which 
manifests itself in finite values of uq. Notice that for the NN 5" = | Heisenberg model on a 
triangular lattice the same approach gives the ground state with the SRO - even for T = 
the frequencies uq remain finite [10] . 

The evolution of the zero-temperature spin-excitation spectrum with p is shown in Fig. 2. 
In the range < p < 0.038 the frequency of spin excitations vanishes at wave vectors Q and 
the dispersion is close to that shown in Fig. 2 (a) (see also Fig. 3). Thus in this range the 
system retains the 120° Neel LRO. Notice that in the analogous classical Heisenberg model 
and in the spin-wave approximation based on this classical solution this phase exists in the 
range < p < 0.11 [li2j. In our approach in the range 0.038 ^ p ^ 0.2 the frequency of 
spin excitations becomes finite everywhere except for k = which points to the destruction 
of the LRO and the establishment of a SRO. A typical dispersion in this range is shown in 
Fig. 2 (b). 

At p ~ 0.2 new zeros appear in the spectrum at the wave vectors Q' = (o, — and (tt, ±:^) 
of the Wigner-Seitz cell [see Fig. 1, 2 (c) and 3]. The appearance of the new ordering vectors 
indicates the establishment of a new LRO phase. The spin-excitation frequencies at these 
wave vectors remain vanishing in the range 0.2 ^ p ^ 0.5 that determines the domain of 
this phase. This phase can be confronted with the two-sublattice metamagnetic phase of 
the analogous classical Heisenberg model, which in our notations exists in the range 0.11 < 
p < 0.5 and has the same ordering vectors [.l,:_2j. This latter phase can be visualized as the 
ferromagnetic ordering of spins along one of the principal directions of the triangular lattice 
and the antiferromagnetic ordering along two others. The two complementary degenerate 
states are obtained from the former by interchanging the directions of the ferromagnetic and 
antiferromagnetic ordering. In our approach quantities averaged over statistical realizations 
are considered, in this instance over the mentioned three degenerate states. Therefore the 
spin correlation (s^^s^) does not depend on the direction of the vector n — m. With this 
remark taken into account one can see in Fig. 4 that in the range 0.2 ^ p ^ 0.5 the signs 
of our calculated spin correlations are in compliance with the discussed ground-state spin 
configurations of the classical model. 

When the frustration parameter p exceeds 0.5 a gap appears at the vector Q' which points to 
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the destruction of the LRO and the estabhshment of a SRO. As seen in Fig. 5, simultaneously 
new minima of the spin-excitation dispersion begin to shape at incommensurate wave vectors 
on the line connecting momenta Q' and their equivalents with the points Qi = (o,±^^^, 

(^±^,±^^^ (see Fig. 1). The frequency of these new minima is small and decreases with 
increasing p. The smallness of the frequency indicates that the correlation length ^ of the 
considered SRO is large. From Eq. one can see that for small temperatures and large 
distances |n — m| the main contribution to the sum is made by a small vicinity of the 
dispersion minima. Using this observation one can obtain an exponential dependence of 
the correlations on the distance and estimate ^ [12]. For p = 0.52 we found ^ ~ 30a and for 
p = 0.55 we determined ^ ~ 50a, where a is the lattice spacing. It should be noted that using 
a large but finite lattice we introduce an upper limit for the correlation length and a lower 
bound for the gap at the ordering vector. For the considered large lattice this gap is extremely 
small in the ordered states and its magnitude is comparable with the accuracy of the used 
optimization procedure. A complication arises when an ordering vector is incommensurate, 
as it happens for p > 0.5. In the general case this vector does not coincide with any allowed 
wave vector for the considered finite lattice. In this instance we are unable to determine 
the exact location of the ordering vector and the size of the gap, as in Fig. 5 for p > 0.65. 
However, the tendency for the decrease of the gap is well seen in this figure and there is 
good reason to believe that the SRO gives way to LRO at p ~ 0.65. The typical dispersion 
of spin excitations for this range is shown in Fig. 2 (d). In the classical version of the model 
this incommensurate LRO phase exists in the entire range 0.5 < p < 1 [T][2] . 

At p ~ 0.96 the ordering vectors reach the points Qi [see Fig. 1, 2 (e) and 3]. The values 
of the frustration parameter p > 0.96 correspond to the case when the interaction between 
NNN spins is much stronger than between nearest neighbors, J2 3> Ji. In this situations the 
lattice is divided into three sublattices with the elementary translation vectors d. Within 
any of the sublattices the interaction between spins is characterized by the large parameter 
J2, while the inter-sublattice interaction is given by the smaller parameter Ji. As a result 
the LRO established for p > 0.96 can be conceived as three interpenetrating 120° spin 
structures on the sublattices. In the limit p — )• 1 spin orientations on the different sublattices 
are independent of one another. This conclusion can be also made from Fig. 4 - in this limit 
Ci, the correlation function between spins in different sublattices, vanishes. At the same time 
Cd, the correlation function between NN spins within the sublattice, tends to the value of 
Ci at p = 0. At the classical level, the LRO with the ordering vectors Qi is only approached 
asymptotically when p — )■ 1. For any p < 1 the incommensurate spiral is lower in energy 
than this former state [I]. 

As seen above, our results for the quantum Heisenberg model reproduce the three LRO 
phases of the analogous classical model and the spin-wave approximation based on this 
classical solution. We found also that in the quantum model these LRO phases are separated 
by phases with SRO. In these transitions, opening and closing the gaps at the ordering wave 
vectors occur with continuity, which is inherent in second-order phase transitions. 

The above results demonstrate that the considered Heisenberg model has an incommensurate 
SRO phase with a large correlation length and the dispersion near the minima, which can 
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explain the quadratic temperature dependence of the specific heat observed in NiGa2S4 [3]. 
However, the respective ordering vectors do not coincide with those found in the crystal. 
Apparently for proper description of the incommensurate phase in NiGa2S4 one has to take 
into account some additional interactions such as a coupling between third neighbors [THfT^ 
or biquadratic interactions [T5] . 

In summary, Mori's projection operator technique was used for investigating the excitation 
spectrum and spin correlations of the two-dimensional S = 1 Heisenberg antiferromagnet on 
a triangular lattice taking into account the nearest- (Ji) and next-nearest-neighbor (J2) in- 
teractions. At zero temperature the competition of these interactions leads to the appearance 
of four second-order phase transitions between phases with long- and short-range magnetic 
order. At the frustration parameter p = J2/ ( Ji -|- J2) ~ 0.038 the ground state of the model 
is transformed from the long-range ordered 120° spin structure with the ordering vectors 
Q = (^,oj and (^!~-^) into a state with short-range ordering. At p ^ 0.2 this latter 

state is changed to the long-range ordered state with the ordering vectors Q' = (o,—^^ 

and (^7r,±^^. This state can be related to the two-sublattice metamagnetic state of the 
classical model. A further transition to a state with short-range order occurs at p ^ 0.5. 
This state has large correlation length and at p ^ 0.65 turns into another long-range ordered 
state. In the range 0.5 ^ p < 0.96 the ordering vectors are incommensurate. With growing 
p they move from the vectors Q' and their equivalents to the vectors Qi = (o, i^^) and 

(^±^, ±^^^, which are reached at p ~ 0.96. The resulting state can be conceived as three 
interpenetrating sublattices with a 120° spin structure on each of them. With p — >■ 1 the 
spin correlations between the sublattices are weakened. The phases with the long-range or- 
der of the quantum Heisenberg model are similar to those of the analogous classical model. 
Additionally the quantum model has two phases with the short-range order which separate 
the long-range ordered phases. 
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Figures 




range < p < 1. For every LRO state only one of several ordering vectors in the zone is shown. 
The small hexagon corresponds to the Brillouin zone of one of three interpenetrating spin sublattices 
with the 120° spin structure which appear in the case p > 0.96. 
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Fig. 2. The dispersion of spin excitations co^ for different values of the frustration parameter p at 
T = 0. 
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Fig. 3. The dependencies of the spin-excitation energies at the ordering wave vectors Q, Q' and 
on the frustration parameter p (symbols correspond to T = and hnes to T/J = 0.2). 
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Fig. 4. The dependencies of the spin correlation functions between the nearest neighbors (Ci), the 
next-nearest neighbors (C^) and (5'n^S'~^2a) frustration parameter p at T = (squares) 

and T/J = 0.2 (Unes). 



12 




Fig. 5. The dependencies of the spin-excitation frequency on the wave vector along the hne Qi — Q' 
for p = 0.52, 0.55, 0.65, 0.71 and 0.95. T = 0. The calculated values of the frequency are shown by 
symbols, lines are interpolations through these points. 
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